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We investigate the ground state of the d^ spin-orbital model for triply degenerate t2g orbitals on a 
triangular lattice which unifies intrinsic frustration of spin and orbital interactions with geometrical 
frustration. Using full or Lanczos exact diagonalization of finite clusters we establish that the 
ground state of the spin-orbital model which interpolates between the superexchange and direct 
exchange interactions on the bonds is characterized by valence-bond correlations. In the absence of 
Hund's exchange the model describes a competition between various possible valence-bond states. 
By considering the clusters with open boundary conditions we demonstrate that orbital interactions 
are always frustrated, but this frustration is removed by pronounced spin singlet correlations which 
coexist with supporting them dimer orbital correlations. Such local configurations contribute to the 
disordered ground states found for the clusters with periodic boundary conditions which interpolate 
between a highly resonating, dimer-based, entangled spin-orbital liquid phase, and a valence-bond 
state with completely static spin-singlet states. We argue that these states are also realized for 
the infinite lattice and anticipate that pronounced transitions between different regimes found for 
particular geometries will turn out to smooth crossovers in the properties of the spin-orbital hquid in 
the thermodynamic limit. Finally, we provide evidence that the resonating spin-orbital liquid phase 
involves entangled states on the bonds. In such a phase classical considerations based on the mean- 
field theory cannot be used, spin exchange interactions do not determine spin bond correlations, 
and quantum fluctuations play a crucial role in the ground states and magnetic transitions. 
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I. SPIN-ORBITAL FRUSTRATION 

Frustration in magnetic systems is usually of geomet- 
rical origin, but it may also arise due to competing ex- 
change interactions.^"^ A common feature of frustrated 
spin systems is that the interactions along different bonds 
compete with one another, and this leads in some cases 
to disordered states and to quantum phase transitions 
when the interaction strength is varied. Another possi- 
bility is so-called " order-by-disorder" , and several micro- 
scopic mechanisms which stabilize ordered state in spin 
systems have been investigated.^"- Here we shall focus 
on the triangular lattice, where frustrated interactions 
suggest that valence-bond configurations in spin model 
with antiferromagnetic (AF) interactions could play an 
important role,^ and we supplement them in this work 
by the frustrated orbital degrees of freedom. 

Recently interesting physical realizations of frustrated 
interactions were introduced in the context of spin-orbital 
superexchange which arises in transition metal oxides 
with active orbital degrees of freedomjii^ In such mod- 
els frustration is intrinsic and follows from the directional 
nature of orbital interactions.^ Therefore, the orbital part 
of the spin-orbital superexchange is frustrated even with- 
out any geometrical frustration. Generic features of this 



direction-dependent interactions are captured within the 
two-dimensional (2D) quantum compass model, which 
exhibits a quantum phase transition through an isotropic 
point with highly degenerate ground state (GS)iii"— 
This high degeneracy is a fingerprint of highly frustrated 
interactions and occurs also in the one-dimensional (ID) 
compass model. It is also characterized by a rather sur- 
prising hidden dimer order in the GS which follows from 
the symmetry of compass interactions 

Frustration in the orbital superexchange models is 
somewhat more subtle — the interactions depend on the 
type of active orbital degree of freedom and in each case 
differ from those in the quantum compass model. In case 
of Cg orbitals the interactions are directional as in the 
quantum compass model, but they are Ising-type only for 
one cubic axis, e.g. for the bonds {ij) along the c cubic 
axis one has the interaction oc <7fcr|, while for the bonds 
{ij) in the ah planes they involve linear combinations of 
{(Tf,af} operators which arise from the directional or- 
bital states along the considered (a or 6) axis^'^- (here 
CT,f and af are Pauli matrices). This particular structure 
follows from the fact that although only one of Cg orbital 
states participates in charge excitations along each sin- 
gle cubic axis and the interactions appear to be classical 
in a ID eg orbital modelfii their superposition is quan- 
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turn either in a 2D modeljis or in a three-dimensional 
oneJ^ In contrast, two t2g orbitals are active and partic- 
ipate in charge excitations along each cubic direction, so 
the respective interactions involve a priori all three com- 
ponents of the orbital pseudospin r = 1/2 doublet, with 
the restriction that the active orbital t2g doublet changes 
with the cubic axis.^*^ For instance, where the degeneracy 
of t2g orbitals is removed by crystal field in the vanadium 
perovskites and xy orbitals are filled, the {yz, zx} orbital 
doublet contributes with orbital fluctuations to the bonds 
along the c axisi^ii^^ 

Realistic superexchange models for perovskite transi- 
tion metal oxides include both orbital and spin degrees 
of freedom, which are strongly interrelated/'^^ Two im- 
portant questions for these models are: (i) whether the 
orbital frustration can be removed by properly selected 
spin states, or frustration is even enhanced by spin- 
orbital quantum fiuctuations, and (ii) to what extent 
spin dynamics may be treated as independent of orbital 
dynamicsj^ The disordered GS was suggested for the t2g 
orbitals in configuration on the perovskite lattice.— 

In the present paper we want to focus on the model 
derived for the transition metal ions in d} configuration 
for the triangular lattice, with frustration being both of 
orbital and geometrical origin. This spin-orbital model 
corresponds to the undistorted NaTi02 and describes 
magnetic interactions in the spin-orbital space, with su- 
perexchange and direct exchange. In the direct exchange 
case the model is exactly solvable and the GS was deter- 
mined by considering the dimer coverings of the lattice, 
with each dimer containing a spin singlet accompanied 
by two active orbitals on the direct exchange bond--^ In 
a general case the GS and the ratio of superexchange and 
direct exchange are not known — the latter depends on 
the respective effective hopping elements via the oxygen 
orbitals responsible for the superexchange and the (dda) 
hopping which gives the direct exchange. Therefore, we 
use it below as a model parameter. A second parameter 
of the spin-orbital model considered here is Hund's ex- 
change interaction. One might expect that also in case of 
superexchange interactions t2g orbitals could order and 
remove the frustration in the triangular lattice, as they 
do, for instance, in LiV02'21 It was argued, however, 
in Ref. [2^ that the GS of the present d^ spin-orbital 
model is disordered and dominated by dimer correlations 
practically for any ratio of the superexchange and di- 
rect exchange interaction. This conclusion was drawn by 
considering the mean-field (MF) states, variational wave 
functions with valence-bond correlations, and exact di- 
agonalization of small systems of not more than iV = 4 
sites. 

The purpose of this paper is to reanalyze the spin- 
orbital states in the d^ spin-orbital model on the trian- 
gular lattice and provide evidence in favor of the dis- 
ordered spin-orbital liquid GS from numerical studies of 
larger finite systems, having up to iV = 10 sites. We use 
extensively Lanczos diagonalization, but for rather small 
systems of size up to = 6 sites where full diagonaliza- 



tion is also possible in the subspace of iS^ = total spin, 
both methods were compared with each other. Thereby, 
we addressed a few general questions which concern the 
spin-orbital physics for varying parameters of the model: 
(i) nature of dimer spin and orbital correlations, (ii) na- 
ture of the transition to the spin-polarized ferromagnetic 
(FM) state with increasing Hund's exchange, (iii) impor- 
tance of spin-orbital entanglement^ and its consequences 
for the transition from low-spin to high-spin states. We 
will provide answers to these questions by considering 
systems of various size and with different boundary con- 
ditions. Altogether, we shall demonstrate that quantum 
fiuctuations determine the GS and the magnetic transi- 
tions to such an extent that classical considerations can- 
not be used in several situations. 



The paper is organized as follows. In Sec. Ill Al we in- 
troduce the d^ spin-orbital model on a triangular lattice 
(as for Ti'^^ or V^^ ions) as derived in Ref. [2^. Basic 
information about the cluster sizes and geometries used 
in Lanczos diagonalization is contained in Sec. Ill Bl The 
numerical results obtained for the isolated clusters of up 
to A^ = 10 sites are analyzed in Sec. IIIII In this section 
we investigate the model in absence of Hund's exchange 
and analyze bond correlations: spin, orbital and spin- 
orbital ones, as well as the orbital occupation. They al- 
lowed us to find certain general trends which are expected 
to determine the behavior of the model in the thermo- 
dynamic limit. The generic transition from quantum to 
classical regime in the singlet sector, with interactions 
evolving from superexchange to direct exchange, is illus- 
trated by a hexagonal cluster in Sec. IIII Al Next we 
consider triangular clusters with open boundary condi- 
tions in Sec. IIII Bl and show that the singlet correlations 
are robust in the entire regime of the exchange interac- 
tions. The results obtained for the clusters with peri- 
odic boundary conditions are reported in Sec. IIII CI In 
Sec. IIV Al we present the orbital model obtained in spin 
polarized case, and investigate the transition from low- 
spin to high-spin states for a few representative clusters, 
presenting the respective phase diagrams in Sec. IIVBI 
Finally, we present the consequences of spin-orbital en- 
tanglement in Sec. IV Al and show that it modifies the 
phase diagrams significantly with respect to those ob- 
tained when spin and orbital operators are disentangled, 
particularly in the regime of purely superexchange in- 
teractions. We also point out in Sec. IV Bl that mean- 
ingful exchange constants cannot be introduced in cases 
where spin-orbital entanglement dominates and stabilizes 
the low-spin ground state with large spin-orbital fluctua- 
tions. General discussion and summary are presented in 
Sec. ED 



II. SPIN-ORBITAL MODEL 
A. Superexchange versus direct exchange 

We consider the spin-orbital model on the triangular 
lattice derived in Ref. [2^ which describes interactions be- 
tween S = 1/2 spins for d} electron configurations, such 
as in NaTi02. The magnetic transition metal ions form 
a triangular lattice for the (111) planes of a compound 
with cubic symmetry. The bonds {ij) are spanning the 
three directions, labeled by 7 = a, 6, c. 

In order to explain the physical content of the model 
we consider a representative bond along the c axis shown 
in Fig. [IJa). For the realistic parameters of NaTi02 
the Sd electrons are almost localized in configurations 
of Ti'^"'" ions, hence their interactions with neighboring 
sites can be described by the effective superexchange 
and kinetic exchange processes. Virtual charge excita- 
tions, d\d}j ^ rfj, between the neighboring sites gener- 
ate magnetic interactions which arise from two different 
hopping processes for active t2g orbitals: (i) the effective 
hopping t = t^ii/ ^ which occurs via oxygen 2pz orbitals 
with the charge transfer excitation energy A and con- 
sists of two tpd steps, in the present case along the 90° 
bonds, and (ii) direct hopping t' which couples the t2g or- 
bitals along the bond and give direct (kinetic exchange) 
interaction. Note that the latter processes couple orbitals 
with the same flavor, while the former ones couple dif- 
ferent orbitals, and there the occupied orbitals may be 
interchanged as a result of a virtual charge excitation. 

For convenience, we introduce the notation 



\a) = \yz) 



\b) = \xz), \c) = \xy), (2.1) 



for the three t2g orbital flavors (colors) , following the one 
used in the perovskite systems?^ adopted here for the tri- 
angular lattice. It follows from the symmetry of orbital 
wave functions that only two of the three t2g orbitals al- 
low for d— p hopping tpd and are active in superexchange 
on any given bond {ij) [Fig. (Ub)], while the remaining 
7 orbitals couple directly along the 7 axis, so they con- 
tribute to the direct (kinetic) exchange, see Fig. [He). In 
addition, each site is occupied by precisely one electron, 
so the density operators satisfy a local constraint at each 
site «, 



nia + no, + nic = 1 



(2.2) 



These symmetry properties on the triangular lattice are 
analogous to those which decide about the form of the ki- 
netic energy for t2g electrons in the perovskite lattice<^i^ 
Local Coulomb interactions at transition metal ions 
are described by two parameters: intraorbital Coulomb 
interaction U and Hund's exchange In the limit of 

large intraorbital Coulomb interaction U intersite charge 
excitations are transformed away and one finds the fol- 
lowing HamiltoniaujS^ 

■H^j[{\-a)ns + ^{l- a)a + a Hd] , (2.3) 




FIG. 1: (Color online) (a) Schematic view of the hopping 
processes between t2g orbitals along a bond parallel to the c 
axis in NaTi02: (i) tpd between Ti{t2g) orbitals and 0{2pz) 
orbitals, with two tpd transitions contributing to an effective 
hopping t, and (ii) direct d — d hopping t' . The hopping t in- 
terchanges two orbital flavors on two sites and contributes to 
the effective superexchange interactions on a bond in the tri- 
angular lattice, while the latter (diagonal) hopping element t' 
contributes to the direct (kinetic) exchange. The t2g orbitals 
are shown by different gray scale (color) and are labeled as 
a, b, and c, see Eq. (|2.ip . In the bottom part the hopping 
processes contributing to (b) superexchange and (c) direct ex- 
change, are shown for the bonds along 7 = a, 6, c axes in the 
triangular lattice. 



where J is the exchange energy. The parameter a is the 
first parameter of the present model Eq. (|2.3p and is 
given by the hopping elements as follows. 



t'- 



t^ + t'2 



(2.4) 



— it interpolates between the superexchange (a = 0) 
and direct exchange (a = 1) limit, as explained in Ref. 
[25I . The second parameter of the spin-orbital model Eq. 
(|2.3p is Hund's exchange. 



U ' 



(2.5) 



It enters the superexchange and direct exchange (see be- 
low) via the coefficients 



1 



ri = 



1 - 377' 



r2 



1 



1 + 277' 



(2.6) 



which follow from the multiplet structure of d^ ions^i 
They correspond to the triplet excitation at energy {U — 
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3Jh), and to singlet excitations at energies {U — Jh) and 
{U + 2Jff). Although the actual values of the Coulomb 
and Hund's exchange elements were deduced from the 
spectroscopic data for Ti^"*" ions corresponding to charge 
excitations by Zaanen and Sawatzky,'^^ U — 4.35 eV and 
Jh = 0.59 cV, which gives a value of 77 ~ 0.14, we shall 
use ?7 as a parameter below in order to investigate the 
transition from low-spin to high-spin states for various 
cluster size and to highlight the difference in the orbital 
correlations in the low and high 77 regime. 

The superexchange part of H can be specified as fol- 
lows 



r2 Si-Sj 



A 



(7) 



;{r2 -rs) S^-S-j - 



and contains two spin operators — a projection on the 
triplet state {Si ■ Sj + |), and an operator —{Si ■ Sj — j) 
which is a projection on the singlet state. These oper- 
ators accompany the coefficients {ri,r2,r3} and express 
the dependence on the excited (f states. 

The orbital operators Aij and Bij in Eq. p.7|) depend 
on the bond direction 7 and involve two orbital flavors 
active in the superexchange, 



0(7) — (rp + rp- ,rp-rp+\_cyrpzrpz 



4"^^;^^ (2.9) 



The operator rii^ in Eq. (|2.7p stands for the number of 
electrons at site i in the orbital inactive in superexchange 
processes, for instance Uj-y = riic in the example depicted 

in Fig. [TJ On the contrary, is the total electron num- 
ber operator at site i for orbitals active in superexchange, 
i.e., in the case shown in Fig. [T]it is nf' — riia -\-nn,. The 
orbital operators {T^,T~,Tl^} refer to the orbital dou- 
blet active in the superexchange on the bond {ij) \\ 7. 
For a single bond, the orbital operators in Eqs. (|2.8p 
and (j2.9p may be written in a very suggestive form by 
performing a local transformation in which the active or- 
bitals are exchanged on one bond site;^ e.g. \a) — ?• \b) 
and |6) — S- \a) on bond {ij) \\ c: 



B 



,(7) ^ 

(7) _ 



2{(T,-Tj 



2 T, 



^''' + i„(7)„(7) 

■ ^"^^ 1 (7) (7) 



(2.10) 
(2.11) 



Here the scalar product in Aij is the conventional ex- 
pression for pseudospin T = 1/2 variables transformed 
as described above, and the product in Bij is the usual 
term which follows from the structure of local Coulomb 



interactions, as well transformed 
follows: 



they are defined as 



(7) 1 / , 

= -(T-+r~ 

2 \ 1-1 31 



11 31 



-T:^T^^{^.12) 



This form follows from the local transformation at site j 
which is introduced for the superexchange in the present 
case." These operators select favored orbital configu- 
rations on two neighboring sites via the Tf^Tj^ terms, 
and orbital fluctuations are described by the T^Tj^ and 

T^Tp^ terms. Note that the z-th pseudospin compo- 
nent is not conserved. For a bond along the axis 7 or- 
bital operators at site i are defined by the electron cre- 
ation {a\, b\^c\\ and annihilation {a^, bi,Ci} operators for 
fermions with a given flavor. For instance, for the bonds 
along the a or 6 axis they are: 





^ib 






Tib 








K 







(2.14) 
n,a). (2.15) 



The direct (kinetic) exchange term involves only vir- 
tual excitations of 7 orbitals on a bond {ij) \\ 7, 



Ha 



E 

i3)\\l 



{ -n(^SrS,+ 

X |^ni-y(l — Tij-y) + (1 — nij)nj-^ 

57} 



Si • Sj 



^{2r2+r,)fsrS,-^ 



Arii^n 



(2.16) 



Therefore the structure of the orbital operators is here 
simpler — they enter as projection operators and give two 
different individual terms either (i) when only one ac- 
tive orbital is occupied, cx ni-y{l — rij^), or (ii) when both 
orbitals are occupied, oc ni-firij^. For the bond shown in 
Fig. [T]7 = c. The structure of excited states is here the 
same as for the superexchange, so the same coefficients 
given in Eqs. (j2.6p occur in both terms, superexchange 
Eq. (12. 7p and kinetic exchange Eq. (|2.16p . 

As explained in Ref. [2^, the two different types of 
hopping processes {t and t') may contribute in a two- 
step virtual djdj ^ d^'fj excitation, in such a way that 
the occupied ortjitals are changed at both sites. In such 
a case the resulting effective interaction are expressed 
in terms of orbital fluctuation operators. For the bond 
shown in Fig. [T] these terms are: 



4 ^ 

{i3)\\c 



ri Si-S-j + 



r2 Si-S-j 



X ( T+T;, + T^TJ^ + T+T+ + Tr^T-, , (2.17) 



where the orbital operators are defined in Eqs. (|2.14l) . 
The form of the Hm'' and Hm terms is obtained from Eq. 
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(I2.17|) by cyclic permutations of the orbital indices. Note 
that these terms describe fluctuations that go beyond any 
static orbital configuration, so they represent corrections 
to the classical treatment of the spin-orbital correlations, 
as discussed in Sees. IV Al and IV Bl 

In the subsequent sections we will focus first on the 
frustrated interactions in the model of Eq. (|2.3p at 77 = 0. 
This case is rather special as the multiplet structure col- 
lapses to a single excitation with energy U (spin sin- 
glet and triplet excitations are then degenerate), and the 
Hamiltonian depends only on the ratio of superexchange 
to direct exchange, parametrized by < a < 1, and has 
the following form: 



(a) N3 



N6 



H 



{'i'j)\h 



2 I Si • Sj 



(7) 1 



,(7)„(7) 



1 



-^(?^^7(l - rij^) + (1 - n,^)nj^^ 



(2.18) 



(7) 



Here the orbital scalar product [Ti ■ Tjj is given by 

Eq. (|2.12p . The Hamiltonian Hq describes AF interac- 
tions between spins, but the orbital terms favor either 
orbital fluctuations (superexchange) or a static configu- 
ration of the same orbitals (direct exchange) on the bond. 
The form of the superexchange oc (1— a) suggests that the 
spin and orbital sectors could be completely equivalent 
and symmetrical at a = for a single bond. However, 
this remains true only as long as active orbitals can be 
selected to contribute to the superexchange as this bond, 
i.e., for n^'^'^n^^'^ = 1, and this equivalence is broken when 
more bonds are considered 

A remarkable feature of the Hamiltonian Eq. (|2.18p is 
the lack of higher symmetry in any of the points when a 
is varied. Even at a = 0.5, where all electron transitions 
have the same amplitude, no higher symmetry occurs as 
the superexchange (a = 0) and direct exchange a result 
from quite distinct processes and cannot be transformed 
into each other. The only analytical solution was found 
in a = 1 case, where at rj = the extremely degenerate 
GS is a liquid of hard-core dimers.— This degeneracy is 
removed at ry > 0, and a valence-bond crystal with a 
large unit cell of 20 sites is formed. 



B. Lanczos diagonalization and cluster size 

In order to establish unbiased results concerning the 
nature of the GS and spin and orbital correlations in the 
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FIG. 2: Clusters (shown in gray) used to investigate the spin- 
orbital model (|2.3p by full or Lanczos exact diagonalization: 
(a) triangular clusters with open boundary conditions N3, N6 
and NIO — here nonequivalent sites and bonds used later are 
indicated by labels 1, 2 and 3, and (b) clusters with periodic 
boundary conditions: rhombic N4, hexagonal N7 and rhombic 
N9. In the latter case positions of neighboring clusters which 
cover the triangular lattice are indicated and all the sites and 
bonds are equivalent (label 1 is used for equivalent sites and 
bonds in N7 and N9 clusters). 



present spin-orbital model Eq. (j2.3p . providing evidence 
in favor of spin-orbital liquid state, we have used diag- 
onalization of finite clusters. As the number of degrees 
of freedom per site is 2 x 3 = 6, the size of the Hilbert 
space increases very fast with increasing system size N . 
The only symmetry which could straightforwardly be im- 
plemented is the conservation of the z-th component of 
total spin. 



N 



(2.19) 



while the orbital state is a priori undetermined. There- 
fore, the use of full exact diagonalization is practically 
limited to system of up to TV = 6 sites, where the size of 
the Hilbert space is 6^ — 46656 and the largest {S^ = 0) 
subspace has the dimension 14580. The use of the Lanc- 
zos method allowed us to investigate systems of up to 
TV = 10 sites. 

Although several other clusters were studied as well, 
we would like to concentrate here only on two classes 
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of clusters which help to identify certain general trends 
for the present spin-orbital model. First of them are tri- 
angular clusters with open boundary conditions (OBC) 
which by construction have nonequivalent sites and are 
expected to favor dimer correlations. They were used to 
identify the dimer correlations and to find their evolution 
with increasing size. These clusters help to understand 
the interrelation between spin and orbital states on the 
bonds which become transparent when the symmetry in 
the orbital space is broken by geometry. The clusters 
considered here contain N — 3, 6, 10 sites and are la- 
beled N3, N6 and NIO, see Fig. [^a). 

The second class of clusters, shown in Fig. Efb), con- 
sists of three clusters with up to 9 sites which cover en- 
tirely the triangular lattice and can thus be investigated 
using periodic boundary conditions (PBC): rhombic clus- 
ter N4, hexagonal cluster N7, and large rhombic cluster 
N9. However, the PBC are not unique and, as we have 
verified, lead to nonequivalent results. Therefore we se- 
lected in each case displayed in Fig. ^h) such PBC which 
result from ordering the considered clusters in rows on 
the triangular lattice, as indicated by the clusters sur- 
rounding the one used for Lanczos (or full) diagonaliza- 
tion. Unlike some other lattice coverings, these bound- 
ary conditions guarantee that all the bonds and sites are 
equivalent in each considered cluster. Therefore, no ad- 
ditional frustration of interactions is introduced by se- 
lecting the PBC and the internal symmetry of the con- 
sidered cluster is preserved. Hence, we suggest that these 
clusters may serve to simulate the situation in the ther- 
modynamic limit. 

C. Correlation functions and entanglement 

In Sees. IHimVl we will compute and discuss the GSs of 
several clusters, by looking at their energies, degenera- 
cies, site occupations, as well as the spin, orbital and 
spin-orbital (four-operator) correlation functions for a 
bond {ij) along 7 axis, given respectively by 

S^j = i ^(n|^, -^jln), (2.20) 

n 

T,, EE i ^(n|(f,.7;)(^)|n), (2.21) 

n 

= ^ E(n|(S',-5,)(f,.f,)('')|n) (2-22) 

n 

n m 

where d is the GS degeneracy, and the pseudospin scalar 
product in Eqs. and (P?^ is defined by Eq. 

(|2.12[) . In clusters with OBC the correlations depend on 
the bond, but for the clusters with PBC all the bonds are 



equivalent and these correlations are uniform. The sum- 
mations include all independent quantum states {\n)} 
which span the possibly degenerate GS manifold. 

The degeneracy d of the GS and the corresponding 
state vectors are easily obtained if full exact diagonaliza- 
tion can be used. On the other hand, due to appearance 
of spurious degeneracies, the Lanczos diagonalization in 
its basic form is not able to quantify reliably the degener- 
acy and to generate the set of suitable GS vectors. In the 
present work we have used the following way to remedy 
this problem: The Lanczos algorithm is performed sev- 
eral times using random initial vectors and a sufficient 
number of the GS vectors is generated. The orthonor- 
malization of this set then yields the degeneracy of the 
GS as the number of independent vectors and the or- 
thonormal GS vectors themselves can be used in the GS 
averaging implied by Eqs. (|2.20p - (|2.22p . Furthermore, it 
is possible to determine the range of possible values of 
the quantities of interest within the GS manifold. This 
is achieved by evaluating the minimum and maximum 
eigenvalue of the matrix representing the corresponding 
operator within this manifold. In such cases the obtained 
ranges of possible values are indicated in all the relevant 
figures in a form of vertical lines. 

The last correlation function Cij Eq. (|2.22p quantifies 
the average difference between the complete spin-orbital 
operator and its decoupled product. Therefore, we use 
it here as the simplest measure of spin-orbital entangle- 
ment. If Cij = 0, the MF decoupling of spin and orbital 
operators on the bond (ij) is exact and both subsystems 
may be treated independently from each other. This 
implies that the GS wave function can be written as a 
product of its spin and orbital parts. For instance, this 
happens for the high-spin states which become the GS at 
large rj^ 

However, we note that the criterion of spin-orbital en- 
tanglement introduced above as Cij ^ can be rigorously 
applied only for systems with nondegenerate GS {d — 1). 
If rf > 1, the averaging introduced in Stj and Tij, used 
in Eq. (|2.22p . means that Cij could be (small but) finite 
even in cases when spin-orbital de facto decoupling takes 
place, and the measure of entanglement would have to 
be more subtle. Nevertheless, we use here Cij Eq. (|2.22p 
as a simple diagnostic tool, and comment in more detail 
on particular cases, where Cij ^ 0. 



III. TOWARD SPIN-ORBITAL LIQUID 

A. Quantum and classical dimers in a hexagonal 
cluster 

In order to understand the consequences of frustration 
on spin, orbital and spin-orbital correlations on the tri- 
angular lattice it is instructive to start with analyzing a 
simpler case of a honeycomb lattice, where the geomet- 
rical frustration is absentj^ A representative cluster for 
such a lattice is a hexagonal cluster H6 (obtained from 
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FIG. 3: (Color online) Evolution of the GS for a hexagon 
H6 with OBC as a function of ain the absence of Hund's ex- 
change (77 = 0): (a) bond correlations — spin Sij Eq. (|2.20|l . 
circles, orbital Tij Eq. H2.2ip . squares, and spin-orbital dj 
Eq. (|2.22p . triangles; (b) orbital electron densities ni~, at site 
i = 1 (left-most site): nia (circles), nib (squares), nic (trian- 
gles). The insets indicate the orbital configurations realized 
in the superexchange limit (q = 0), for 0.44 < a < 0.63, and 
in the direct exchange limit (a = 1). The vertical lines in- 
dicate the range of possible values of the particular quantity 
(bond correlation or electron density). They should not be 
confused with any kind of numerical error as they indicate an 
exactly determined range due to the GS degeneracy. 



the N7 cluster in Fig. [IJb) by removing the central site) 
which will be considered here with OBC. It serves to un- 
ravel the generic behavior of the orbital correlations for 
increasing a and to investigate the spin-orbital entangle- 
ment in different parameter regimes of the spin-orbital 
model Eq. (12.31) . This cluster was analyzed using Lanc- 
zos diagonalization, and we present also the degeneracy 
of the GS below. 

One finds that all the orbitals contribute equally in the 
entire range of a, and each orbital state is occupied at two 
out of six sites in the GS, in the entire regime of a. How- 
ever, the orbital state changes under increasing a and one 
finds four distinct regimes by analyzing the evolution of 
the GS shown by the correlation functions and orbital 
densities displayedin Fig. [31 The transitions between 
them are abrupt (first order) and occur by level cross- 
ing. Each site in the hexagonal cluster participates only 
in two bonds which breaks the spatial symmetry in the 
orbital space in a particular way. In the superexchange 
model at a = there is precisely one orbital at each site 
which contributes to the interactions along both bonds, 
and we have found that indeed — 1 for this particu- 
lar orbital [for site i = 1, c orbital is active along both 
bonds, as shown in Fig. [IJb)]. This results in a unique 



GS which is characterized by a frozen orbital configura- 
tion and triplet orbital correlations at each bond 
i.e., T-^- — 0.25, see Fig. [2Ia). Under these circumstances 
the orbitals decouple from spins and Eq. (I2.18[) reduces 
to the AF Heisenberg model on this cluster. Therefore, 
the GS is disentangled, with Cij ~ 0, and one finds that 
the spin correlations are exactly the same as for the ID 
chain of iV = 6 sites with PBC, i.e., Sij = -0.4671. 

Orbital fluctuations gradually increase with increasing 
a, but the above GS remains stable up to a ~ 0.10. 
For larger 0.10 < a < 0.44 the fluctuations "soften" the 
orbital state and allow for local fluctuations along the 
bonds. As spins are also involved, this change of the 
GS is not gradual but occurs as a quantum transition to 
the state with degeneracy d ~ 2. In this GS the spin 



correlations weaken to Si, 



-0.27 and the joint spin- 



and-orbital fluctuations contribute with flnite mixed cor- 
relation function, Cij ~ —0.12, see Fig. [3fa). 

The degeneracy of the GS results in different values 
of the possible orbital occupancy at each site — while 
the probability of occupying the orbital c at site 1 is now 
reduced to nic = 2/3 in each component, the remaining 
orbital with flnite density is either a or 6, depending on 
the actual wave function. Each component satisfies the 
local constraint given in Eq. (|2.2p . As a result, the aver- 
age density nia = nn, = 1/6 follows from two contribut- 
ing eigenfunctions, with either nia = 1/3 or nu = 1/3, 
and the third orbital empty {nib = or nia = 0). This 
is marked in Fig. |3^b) by the vertical lines (error bars) 
which indicate the range of possible values for the rele- 
vant electron density ni^. 

In agreement with intuition, when a ~ 0.5 and all 
interorbital transitions shown in Fig. [T] have equal am- 
plitude, there is large orbital mixing which is the most 
prominent feature in the GS found in the intermedi- 
ate regime of 0.44 < a < 0.63. It is not centered at 
a = 0.5 as there are less processes which contribute to 
direct exchange than to superexchange, the energy gain is 
lower in the direct exchange regime and thus this regime 
which comes next is narrower. Although both superex- 
change and direct exchange suggest AF spin couplings, 
the orbitals fluctuate here strongly and couple to the 
spins. Therefore, AF spin correlation function is again 



reduced to Sij 
correlations, Ti, 



-0.21. Remarkably, both the orbital 
—0.18, and the mixed correlations, 
Cij ~ —0.13, are also negative, and the GS is unique 
(d = 1). Here all the orbitals contribute equally and 
niy — 1/3, as seen in the inset of Fig. Efb). We recog- 
nize this state as a prerequisite of the spin-orbital liquid 
state which dominates the behavior of the triangular lat- 
tice, as we demonstrate below in Sec. IIII CI Actually, 
there is also some similarity between this fluctuating GS 
and the GS of the ID SU(4) spin-orbital model,— but 
here the symmetry is lower by construction. 

The regime of larger values of a > 0.63 favors the di- 
rect exchange interactions, supported by pairs of identi- 
cal orbitals active on an exchange bond. Having only one 
orbital flavor active along each bond, only three bonds 
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may be occupied by spin singlets and contribute with a 
direct exchange energy, and there are two distict config- 
urations with differently distributed orbital occupations 
along the hexagonal ring, shown in Fig. Elb) . These two 
distinct GSs (degeneracy d = 2) cause again two distribu- 
tions of the orbital densities, this time varying between 
and 1 in the majority of the direct exchange regime, 
i.e., for a > 0.7. After averaging over two degenerate 
states, the average occupancy for the orbitals which are 
active on one of the bonds originating from each site i 
is riia — Tin, = 0.5. However, there is also a narrow 
range of 0.63 < a < 0.7, where such fluctuations have a 
lower amplitude, only between 1/3 and 2/3. This sug- 
gests that the orbital fluctuations play still an important 
role here and couple weakly since Cij ~ 0.05, in contrast 
to a > 0.7 characterized simply by two distinct orbital 
configurations and factorized spin and orbital degrees of 
freedom, i.e., Cij = 0. Indeed, as a result of averaging 
over two degenerate wave functions in the GS, see inset 
of Fig. [nijb), one finds at a = 1 the bond correlations 



N6 



-3/8 and = 1/8. These results demonstrate 



that two orbital configurations are static. 



Triangular clusters with open boundary 
conditions 



In contrast to the hexagonal cluster considered above, 
triangular clusters of Fig. [^fa) are characterized by frus- 
trated spin-orbital interactions and contain nonequiva- 
lent bonds. Therefore the case of decoupled spin and or- 
bital dynamics cannot be realized in the superexchange 
limit (at a = 0) in none of the clusters. In fact, the 
orbitals try to adjust themselves to the frustrated ge- 
ometry, but in general several equivalent configurations 
contribute and the obtained results follow from averag- 
ing over them. The smallest triangular cluster N3 was 
already analyzed in Ref. '25', we therefore concentrate 
here on N6 and NIO. Below we discuss first the bond 
correlation functions and next explain them by present- 
ing the electron distribution over t2g orbital states. 

The intersite spin, orbital, and spin-orbital correlations 
are presented in Fig. |4] for two classes of bonds shown 
in Fig. ma): a bond involving a corner site labeled as 
1, and an internal bond being close to a corner labeled 
as 2. In order to simplify the notation we label the re- 
spective bond correlation functions for a bond n = 1, 2 by 
a bond index as iS„, Tn, and C„, respectively. The data 
points indicate three physically different regimes (similar 
to some extent to 116 cluster of Sec. IIII A[) : (i) the su- 
perexchange regime in a range of small values a > 0, (ii) 
an intermediate regime for values of a close to but typ- 
ically larger than a — 0.5, and (iii) the direct exchange 
regime far large values of a close to a = 1. The range 
of values of a for each of these three regimes depends 
on the cluster size, but certain common features can eas- 
ily be recognized in Fig. |31 Altogether, the intermediate 
regime where the electrons are almost equally distributed 
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FIG. 4: (Color online) Bond correlations for triangular clus- 
ters for increasing a in absence of Hund's exchange (?; = 0), as 
obtained for topologically equivalent bonds n = 1, 2 in trian- 
gular clusters N6 (top) and NIO (bottom) with OBC, see Fig. 
2(a): spin (5„, circles), orbital {Tn, squares), and spin-orbital 
(Cn, triangles) correlations. 



and spin-orbital fluctuations are strong becomes broader 
by increasing cluster size from N6 to NIO. 

The clusters N6 and NIO do not have any spin degen- 
eracy and total spin state is a singlet 5 = in the en- 
tire parameter range. The degeneracy of N6 cluster (see 
Table I) follows therefore from the distribution of three 
spin singlets over the cluster accompanied by matching 
the orbital state in such a way, that the energy of these 
singlet states can indeed contribute to the GS energy. 



TABLE I: Degeneracy of different GSs found in the spin- 
orbital model Eq. (|2.18|) at the superexchange point (q = 0), 
at finite but small a = e, in the intermediate regime (a = 0.6), 
close to (1 — Of = e) and at the direct exchange (a = 1) point 
for the triangular N3, N6 and NIO clusters shown in Fig. 2(a). 
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FIG. 5: (Color online) Orbital occupations ni^, in triangular 
clusters for increasing a in the absence of Hund's exchange 
(77 = 0): at sites i = 1,2 for N6 cluster (top), and sites 
i — 1,2,3 for NIO cluster (bottom), see Fig. [2fa). Insets 
show two representative orbital occupancy distributions for 
the superexchange (left) and direct exchange (right) regime. 



The overall tendency towards singlet spin correlations 
on the bonds originating from a corner site 1, see Fig. 
mja), is observed in both N6 and NIO clusters, where 
Si < —0.12 in the entire regime of a. Gradual localiza- 
tion of spin singlets near corner sites with increasing a is 
suggested by the decreasing values of iSi , with the lowest 
values reached at a = 1 in both cases. The superex- 
change regime extends up to a ~ 0.54 {a ~ 0.51) for N6 
(NIO) cluster, while the direct exchange dominates for 
a > 0.64 {a > 0.67), respectively. A small increase of 
Si in N6 cluster for < a < 0.54 could be understood 
as a consequence of gradual increase of orbital fluctua- 
tions towards the intermediate regime, where spin and 
orbital dynamics start to decouple from each other. This 
trend is recognized from simultaneous decrease of spin 
correlations Si and increase of spin-orbital correlations 
Ci. A common feature for both N6 and NIO clusters in 
the regime of large a is vanishing spin-orbital correlation 
function (Ci = 0), see Fig. |31 

We remark that also the correlations on bonds labeled 
as 2 in the clusters N6 and NIO, see Fig. 2(a), have sev- 
eral common features. First, spin correlations are nega- 



tive (1S2 < 0) in the entire range of a except for a = 1, 
where one finds S2 = 0- We argue that this result mani- 
fests frustration of the internal bonds (labeled 2) in both 
clusters in contrast to the bonds originating from a cor- 
ner (labeled 1). Second, the orbital correlations T2 are 
positive both for small and large values of a, while in 
the intermediate regime these correlations are negative 
due to large orbital fluctuations. Finally, the spin-orbital 
correlations vanish on bonds 2 in both clusters when the 
direct exchange point a = 1 is approached. These re- 
sults confirm the observations made above for the bonds 
1 originating from a corner that spin-orbital entangle- 
ment is absent at the direct exchange point. 

Three different regimes of spin and orbital correlations 
recognized in N6 and NIO clusters are characterized by 
quite different density distributions of electrons which 
obey the local constraint Eq. (|2.2p . The electron den- 
sities, shown in Fig. [SJ confirm the observations made 
above by analyzing the intersite correlations that the GSs 
of both clusters can be classified as belonging to three dif- 
ferent regimes dominated by: (i) the superexchange, (ii) 
orbital fluctuations, and (iii) direct exchange. Different 
role played by the orbitals {a, b, c} along particular direc- 
tions in N6 cluster are highlighted by the two insets in 
Fig. [5] The bonds which originate at a corner site i — 1 
are remarkably similar to each other. The common active 
orbital c for the superexchange along both bonds has a 
large electron density, while the remaining orbitals may 
contribute only along one of these bonds. In the smaller 
N6 cluster the density distribution at a = amounts to 
nic ~ 0.74 and nia = nib — 0.13. This density distri- 
bution cannot be easily deduced by analyzing the degen- 
eracy of the GS (Table I) and by averaging spin singlet 
configuration over the bonds in the N6 cluster which gives 
instead {nia,nih,nic} = {1/6,1/6,2/3}. Therefore, we 
find here the first example where the orbital fluctuations 
play a role and modify the electron density distribution 
with respect to the classical expectations. Nevertheless, 
the spin singlets are accompanied by appropriate active 
orbitals along three isolated bonds, each of them origi- 
nating from one corner site of N6 triangle. This is also 
confirmed by the density distribution found for the site 
i = 2, where one finds the density of 0.5 for both orbitals 
active along the bonds on the triangle edge and for 
the third orbital which contributes to the superexchange 
along the other two triangle edges. To some extent this 
is also observed in the larger NIO cluster. 

The density distribution at sites i = I and i ^ 2 
changes towards more isotropic one for increasing a, but 
the symmetry in the orbital space is always broken by 
geometry and the densities differ somewhat even in the 
intermediate regime for a 0.6 (Fig. [S]). For larger 
values of a one recognizes easily the situation of singlet 
spin dimers distributed over the cluster. Each dimer is 
based on a single orbital fiavor active in the direct ex- 
change on the particular bond. This state is surprisingly 
robust against orbital fluctuations in N6 cluster, where 
only the orbitals a and b are occupied at site i = 1, and 
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n-ia = nit, = 0.5. Actually, a similar situation is found 
also in NIO cluster, but here the GS is degenerate, see Ta- 
ble I, and a range of possible electron densities {nia, nit} 
is found instead. Altogether, we have found that nic = 
in all the considered triangular clusters when the direct 
exchange limit is approached. 

The distribution of orbital densities at sites labeled 
i = 2 in clusters N6 and NIO in the regime of large 
a > 0.7 demonstrates that indeed these clusters are dom- 
inated by the spin singlets touching a corner site each. It 
is for this reason that the density approaches n2a = 1 
when a —7- 1, and the other two orbitals are empty. In 
NIO cluster the density n2a is somewhat reduced due to 
the geometric constraints and additional frustration in- 
troduced by the total number of five spin singlets which 
implies several equivalent states with different orbital 
density distributions in the cluster. 

The only site in the triangular clusters which recovers 
full symmetry in the orbital space is the central i = 3 
site of NIO cluster. Here we have found that all three 
orbitals contribute equally in the GS in the entire range 
of a, with n^^ — 1/3 (Fig. [5]), but certain fluctuations 
around this average value are observed both in the su- 
perexchange (a < 0.52) and direct exchange {a > 0.66) 
regime. This result may be treated as a precursor of 
the situation encountered in the infinite lattice, where 
the geometrical frustration favors the disordered state. 
Such an interpretation is also supported by the fact that 
the GS energies per bond (not shown) systematically in- 
crease with increasing cluster size (for fixed a) among 
N3, N6 and NIO clusters, particularly close to superex- 
change (a — 0) and direct exchange (a = 1) points. We 
shall investigate this situation in more detail below (in 
Sec. nil C|) by considering the clusters with PBC. 



C. Clusters with periodic boundary conditions 

After analyzing the triangular clusters with OBC in 
Sec. IIIIBI we turn to the clusters N4, N7 and N9 with 
PBC, shown in Fig. [Hb), which have all the sites equiva- 
lent and are thus representative for the triangular lattice 
in the thermodynamic limit. The intersite correlation 
functions obtained for N4 cluster were analyzed in Ref. 
m, and we shall concentrate here on both larger clusters, 
N7 and N9. 

Unlike for the triangular clusters where abrupt tran- 
sitions between distinct regimes of particular spin and 
orbital correlations were found, one finds here that the 
intersite spin, orbital and spin-orbital correlations evolve 
continuously with increasing a for N7 and N9 clusters, 
and no distinct regimes with dominating either superex- 
change or direct exchange can be identified. In case of 
N7 cluster the spin correlations are AF and constant, 
Sij ~ —0.11. Note that this value is somewhat higher 
than the average obtained by considering randomly dis- 
tributed spin singlets over the triangular lattice, i.e., oc- 
cupying every sixth bond in the lattice and leading to 
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FIG. 6: (Color online) Evolution of the frustrated GS for N7 
cluster with PBC shown in Fig. [U^b) with increasing a in the 
absence of Hund's exchange: (a) intersite bond correlations 
— spin {Sij, circles), orbital {Tij, squares), and spin-orbital 
{Cij, triangles); (b) orbital occupations ni~, per site (at a 
representative site i = 1). 

{Sij) ~ —0.125. However, the obtained smaller value can 
be justified as follows. In the low-spin phase one has 
5=1/2 total spin, and one can determine the intersite 
spin correlations using the following identity; 

= 7Sf + 42{S^- Sj) . (3.1) 

For this cluster every pair of sites {i, j} is a nearest neigh- 
bor pair and forms a bond (due to PBC), so Eq. p.ip 
follows. The value {Si ■ Sj) = -3/28 = -0.107 obtained 
from it is much reduced from the classical AF spin corre- 
lations — 1/4 in the Neel state on a square lattice, which 
is a consequence of high frustration of the triangular lat- 
tice. Note that spin correlations are here substantially 
reduced by the geometrical frustration, and seem not to 
be further hindered by orbital fluctuations, unlike in the 
ID SU(4) modelj.22, where the coupling to the orbital cor- 
relations is crucial and reduces spin correlations although 
the geometrical frustration is absent. 

For the values of a < 0.8 the orbital correlations Tij 
are even a bit lower than the spin ones, and decrease 
somewhat with increasing a for a < 0.6. We emphasize 
that both spin and orbital correlations are here negative, 
so the present spin-orbital model on the triangular lat- 
tice violates the Goodenough-Kanamori rules that these 
correlations should be complementary."^^ This is also re- 
flected in the finite spin-orbital correlations Cij ~ —0.07 
which indicate that entangled states play an important 
role in this parameter range, see Fig. [S] When a in- 
creases further, however, the orbital state is reorganized 
and the orbital correlations rapidly decrease. At the 
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same time the mixed spin-orbital correlations also de- 
crease suggesting gradual disentanglement of spin and 
orbital degrees of freedom. Finally, aX a — 1 the spins 
decouple from the orbitals, which agrees with the GS 
consisting of uncoupled spin singlets distributed over the 
triangular lattice;^ 

As all the orbitals are equivalent, the average occu- 
pancy is rii^ = 1/3, see Fig. |6l However, apart from 
the spin degeneracy due to the total spin state S ~ 1/2 
realized for the cluster with odd N = 7 sites, one finds 
that several equivalent orbital configurations contribute, 
with one of the orbitals occupied by three electrons and 
the other two by two each. For this reason we have found 
degeneracy d = 6 x 2 = 12, see Table II and large fluctu- 
ations of the density distribution, typically between 0.1 
and 0.6 for low values of a < 0.5, and increasing towards 
a — 0.85. Next the quantum nature of the GS makes it 
only spin degenerate and d — 2 for a > 0.85 and no fiuc- 
tuations in the orbital occupancies were found. Finally, 
in the direct exchange limit a = 1 independent dimer dis- 
tributions over the cluster dominate and determine large 
orbital degeneracy d = 147 x 2 of the GS. This degener- 
acy due to the orbital distribution over the cluster can 
be understood as given by 3 possibilities of having one 
dominating orbital fiavor, 7 positions of this extra flavor 
in the cluster, and 7 possible distributions of singlets over 
the cluster, i.e., 3x7x7 = 147. 

The second cluster with PBC contains = 9 sites 
and is better designed to study the present model as 
the degeneracy of the GS is expected to be lower (ex- 
cept at a = 1). In this case electrons can be equally 
distributed over the orbitals, and three electrons occupy 
each of them. Fluctuations over different values of bond 
orbital correlations Tij and differently occupied orbital 
states, see Figs. [71[a) and[71[b), cannot be avoided within 
the cluster, but they are typically smaller than those 
found for N7 cluster. All average values for the inter- 
site correlations are rather similar to those found in N7 
cluster for small a, with Sij ~ —0.90, Tij ~ —0.97, and 
Cij ~ —0.64 at a = 0. Here all the orbital states are 
equally populated as shown by the inset in Fig. [71 but 
undergo local fiuctuations, seen both in the intersite cor- 



TABLE II: Degeneracy d of the GSs found for the d^ spin- 
orbital model Eq. (|2.18|l at the superexchange point (a — 0), 
in the intermediate regime (0 < a < 1) and at the direct 
exchange point (a — 1) obtained for N7 and N9 clusters with 
PBC. The first factor in d gives the orbital degeneracy which 
is multiplied by spin degeneracy 2 for the GS with 5 = 1/2 
total spin. 
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FIG. 7: (Color online) Evolution of the frustrated GS for N9 
cluster with PBC shown in Fig. [U^b) with increasing a in the 
absence of Hund's exchange: (a) intersite bond correlations 
— spin {Sij, circles), orbital {Tij, squares), and spin-orbital 
{Cij, triangles); (b) orbital occupations ni-y per site (at a 
representative site i = 1). Insets show typical orbital patterns 
in the superexchange {a — 0) limit and direct exchange {a = 
1) limit. 

relation functions {Sij, Tij, Cij} and in the electron den- 
sities {nia,nih,nic}- 

The range of fluctuations in bond correlations is re- 
duced but stays finite up to a ~ 0.41. Here the orbital 
degeneracy of the GS is first 6 up to a = 0.26, and next 
drops to 4, see Table II. This regime is followed by a quali- 
tatively new situation in the intermediate regime of a val- 
ues (compared to N7 cluster), with no fluctuations in the 
orbital distribution when 0.41 < a < 0.85. This regime 
is characterized by low degeneracy 2 in the orbital space. 
In this case the orbitals undergo strong local quantum 
fluctuations but their distribution in the cluster does not 
change, as seen in the stable density distribution, with 
nij = 1/3. This regime can be identified as dominated 
by orbital fluctuations due to the mixing terms H„i Eq. 
(|2.17|) . with gradual suppression of spin-orbital fluctua- 
tions under increasing a seen in the reduced values of 
|C.,|. 

Finally, when the orbital mixing terms sufficiently de- 
crease, one finds that both spin and orbital correlations 
fluctuate stronger above a = 0.85 which follows from a 
random electron distribution over the available orbital 
states. In contrast to the superexchange regime with 
equally distributed electrons over the orbital flavors, the 
representative state shown in the inset in Fig. [71 is dom- 
inated by c orbitals. Equivalent conflgurations can be 
obtained by cyclic permutations of the density distribu- 
tion in the {a, b, c} orbitals, and changing favored dimer 
direction in the cluster. For this reason degeneracy of 
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FIG. 8: (Color online) Ground state energy Ea per bond for 
clusters N4 (filled squares), N7 (diamonds), and N9 (circles) 
with PBC conditions for increasing a. For a comparison the 
result obtained for N7 cluster with OBC is shown by empty 
squares. Parameter: r; = 0. 



the GS increases by a factor of 3, reflecting three possi- 
ble with states dominated by one orbital flavor, see Table 
II. The direct exchange limit is again special, and char- 
acterized by large orbital degeneracy 756. 

The GS energies obtained for N4, N7 and N9 clus- 
ters with the PBC (Fig. [5]) exhibit a monotonous in- 
crease with increasing a. The energy is the lowest for 
the smallest N4 cluster, and increases when the cluster 
size is increased to N7 or N9, indicating frustrated in- 
teractions. However, the energies obtained for N7 or N9 
clusters are very close to each other which we take as an 
evidence that these clusters are already representative 
for the situation in the thermodynamic limit. We re- 
mark that the energy obtained for N7 cluster with OBC 
is lower than that found for the same cluster size with 
PBC which is again consistent with strongly frustrated 
spin-orbital interactions on the triangular lattice. This 
resembles the situation in spin systems with frustrated 
interactions, where one expects that clusters with OBC 
would give lower energy per bond than the ones with 
PBC as then additional breaking of symmetry is possible 
in the GS. 



IV. THE MODEL AT LARGE HUND'S 
EXCHANGE 

A. Orbital model for the FM phase 

Until now we considered the spin-orbital model Eq. 
(|2.3p in the regime of no Hund's exchange, r\ — Fi- 
nite Hund's exchange is responsible for the competition 
of FM with AF spin interactions in the perovskite Mott 
insulators described by similar spin-orbital models, and 
the orbital order is usually modified in a particular way. 
Well known examples are the A-AF phase in LaMnOaj^^ 
or C-AF phase in LaVOsfSi The present and earlier?^ 



study of the spin-orbital d}- model on the triangular lat- 
tice, however, suggest that orbital disorder is favored in 
this geometrically frustrated lattice. Having no orbital 
order excludes de ]acto an intermediate phase (at inter- 
mediate values of r\) with coexisting FM and AF interac- 
tions in the present case. This is confirmed by the exact 
diagonalization of finite clusters, at least in the consid- 
ered case when all the interactions along three bond di- 
rections are equivalent. Thus, there are two problems 
to be addressed in the theory: (i) the nature of orbital 
correlations in the FM phase in the range of large values 
of 77, and (ii) the phase diagram in the {pi^r\) plane. The 
first of these questions is easier to answer and we consider 
it below, the second one will be discussed in Sees. IIVBI 
and El 

The exchange interactions in H given by Eq. (|2.3p 
simplify in the FM phase, when only excitations to high- 
spin states contribute and all the low-spin terms vanish, 
i.e., {Si ■ Sj — 4) = 0. The Hamiltonian reduces then to 



-'J 4 
the orbital model 



Horb= -^Jri ^ I - (1 - a) 



2A[? + (n,^+nj^)-2 



+ ^a{l - a) {T+T+ + T+T+ + Tr^Tf^ + T^rTf^ 
nij{l - Ujj) + {I ~ ni^)nj^ |. (4.1) 



a 



In the superexchange regime (a ~ 0) it favors pairs of 
different orbitals, both not oriented along the considered 
bond, and in the direct exchange regime (a ~ 1) — pairs 
of different orbitals, one oriented along the bond and the 
other not. 

Complete information about the orbital correlations in 
the parameter regime where the FM phase is stable may 
be obtained by considering the following orbital projec- 
tion operators for a bond (ij) \\ 7: 



p(7) ^ 
r(7) _ 



{n,^nj^) , (4.2) 
{n,^{l - Uj^)) + ((1 - n,^)nj^) , (4.3) 
((l-n,;^)(l-n„)). (4.4) 



Here the operator rii^ stands for the electron density in 
the orbital oriented along the bond (ij) at site i, while 
(1 — rij^) is the complementary electron density in the two 
remaining orbitals, as given by the local constraint Eq. 
(|2.2p . The above projection operators may be treated as 
probabilities to encounter a given orbital configuration, 
and they obey the usual condition. 



3(7) 



Q 



(7) 



(4.5) 



Knowing these probabilities allows us for a complete 
characterization of the orbital state on a representative 
bond {ij) II c. 

First we present distinct differences between the dis- 
tribution of occupied orbitals between the superexchange 
(a ~ 0) and direct exchange {a ~ 1) regime found for 
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FIG. 9: (Color online) Evolution of the orbital state in the FM 
GS of the hexagonal H6 cluster with OBC, as found at large 
ri = 0.2 for increasing a: (a) spin S = Sij, orbital T = Tij 
and spin-orbital C = Cij bond correlations; (b) orbital bond 
projection operators P = P^^^ (circles), Q = Q^j' (squares), 

and R = 7?^j' (triangles); (c) orbital electron densities ni-y 
on the most left cluster site i = 1. Insets in (c) show two 
equivalent states in the superexchange (a = 0) and direct 
exchange (a = 1) limit. 



the FM hexagonal H6 cluster. Here the orbital bond cor- 



relations change from T^j 



-0.34 at a = to T., 



at 



a = 1, while the spins are FM (Sij = 0.25) and disentan- 
gled from the orbital state {Cij = 0), see Fig. This 
latter feature is common for FM states in other clusters 
as well (see below) and the Goodenough-Kanamori rulSiSi 
is obeyed, i.e., FM spin correlations are accompanied by 
negative orbital correlations which indicate that orbitals 
show a tendency towards alternating orbital order. 

The FM state in the superexchange regime {a ~ 0) is 
stabilized by pairs of orbitals which do not include the 
one oriented along the considered bond. This is indicated 
by the inset in Fig. Hfc), where two possible configura- 
tions with pairs of active orbitals on the bonds of H6 clus- 
ter, forming almost orbital singlets, are shown. In this 
case the largest average projection operator is see 
Fig. Elb), and the orbital density is the largest for the c 
orbital which participates in the singlets for both possible 



cluster coverings, i.e., nic = 0.5 and nia = nih = 0.25. 

The intermediate regime extends in the FM H6 cluster 
from a = 0.19 to a = 0.77, where the orbital densities 
are the same in each orbital, nij = 1/3, see Fig. [DJc). 
However, the orbital correlations on the bonds are not 
constant, but gradually change towards pairs of one 7 
orbital accompanied by a different orbital on each bond 
(ij) II 7 with increasing a, as depicted by the projection 

operator (Qj-J'') in Fig. ^h). Simultaneously the orbital 
correlation Tij is gradually reduced. Finally, for a > 0.77 
the orbital distribution changes to pairs of orbitals, one 
oriented along the bond and the other not, favored in 
the direct exchange regime. Once again, there are two 
possible distributions of orbitals over the H6 cluster, and 
this gives for site i = 1 two contributing densities: nia = 
nib — 0.5, while the orbital which participates in the 
direct exchange only on two horizontal bonds is empty, 
i.e., nic = 0, see Fig. (He). 

The above transparent picture of the occupied orbitals 
in different a regimes for H6 cluster is strongly modified 
in the GS of the symmetric N7 cluster by frustration of 
spin-orbital interactions and by the absence of symmetry 
breaking in the orbital space due to the PBC, see Fig. (TU] 
At a = one finds negative orbital correlation function 
Tij ~ —0.23, see Fig. fTOFa). The orbital correlations are 
quite well developed here as they are not hindered by 
spin fluctuations, as it was the case at 77 = (Fig. O. 
With increasing a the orbital correlations are gradually 
reduced, and reach the limiting value Tij = —0.0357 at 
a = 1 which corresponds to randomly distributed dimers 
consisting of a pair of active and inactive orbital in the 
direct exchange over the bonds of N7 cluster. 

Two distinct regimes: one with fluctuating values of 
Tij for a < 0.85, and the other one with uniquely de- 
termined Tij for a > 0.85 correspond to the degenerate 
and nondegenerate GS of the N7 cluster, respectively. 
Similar fluctuating results for the average projection op- 
erators {Pj;j'\Q['^\ R^'J^} and for the orbital densities 
{nia, nib, riic} were found in the range of a < 0.85, see 
Figs. [TUf b) andfTUTc). In the regime of small a < 0.4 con- 
figurations with two orbitals active in the superexchange 
processes are the most probable ones, i.e., i?^-^-' > pj;^'' 

and iJ^J-* > Q[]^ ■ On the contrary, when a > 0.4, pairs 
of orbitals, one active and the other one inactive in the 



direct exchange processes, dominate, i.e., Q-J^ > R 



and P 



(7) 



?(7) 



when a 



. „ „ ,1. Here the results suggest 

a single quantum state with equally distributed orbital 
flavors over the cluster, ni^ = 1. 

We have verified that the above evolution of the or- 
bital state in N7 cluster is representative for the present 
triangular lattice in the FM regime by considering two 
larger clusters with PBC (not shown): (i) N12 cluster 
obtained by adding 5 sites to N7, e.g. on the right hand 
side and on top, and (ii) a star-like N13 cluster obtained 
by adding a triangle to each side of N7 cluster. Both clus- 
ters have all sites equivalent and may be used to cover the 
lattice (in the second case two equivalent coverings differ 
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FIG. 10: (Color online) Evolution of the orbital state in the 
FM GS of N7 cluster for increasing a, found with PBC at large 
ri = 0.2: (a) bond orbital correlations T = Tij; (b) orbital oc- 
cupation correlations P = P^J^ (circles), Q = Q'J' (squares), 
and R = R'i'-^ (triangles); (c) orbital electron densities ni-y. 



by chirality). In both cases we found that the orbital 
bond correlations Tij increase from Tij ~ —0.23 at a = 
to Tij ~ at a = 1. The occupied orbitals give again 

(as for N7) large ~ 0.5 at the superexchange limit 
a = 0. It decreases with increasing a, and one finds in- 
stead large Q-J^ ~ 2/3 at the direct exchange case a = 1. 
Interestingly, the orbital fluctuations shown by the verti- 
cal lines are reduced in both cases, and they do not occur 
at aU in N12 cluster both for a < 0.62 and for a > 0.81. 



B. Phase diagrams 

Perhaps the most intriguing question concerning the 
GS of the spin-orbital model Eq. (|2.3p is the phase di- 
agram and the way the FM state occurs as a function 
of both model parameters, a and 77. Having no possibil- 
ity to access the phase diagram in the thermodynamic 
limit (see also Sec. |V|, we shall concentrate here on a 
few representative clusters trying to extract the common 
and generic features of the low-to-high spin transition. 

The hexagonal H6 cluster, with low-spin and high-spin 
states shown in Figs. |3]and|ni serves here as an example 
of unfrustrated geometry. The transition to the high- 



0.24 




J S=2 ' 


1 




0.22 


- S=1 




S=3 


- 


0.20 








- 


0.18 










0.16 




S=0 




Jl ? 


0.14 










0.12 








' S=2 _ 



0.0 



0.2 



0.4 0.6 

a 



0.8 



1.0 



FIG. 11: (Color online) Phase diagram in the (a, ry) plane as 
obtained for the hexagonal H6 cluster with OBC. The total 
spin of the GS is indicated in the transition regime from the 
singlet phase (<S = 0) to the high-spin (<S = 3) phase. 



spin iS = 3 state is then gradual and passes through 
intermediate S = 1 and (in some cases) 5 = 2 states 
when a is close to or 1, see Fig. [TTJ In contrast, for 
the intermediate values of 0.22 < a < 0.62 the transi- 
tion takes place directly between the S = and 5 = 3 
states. We recall that in the regime of intermediate a val- 
ues orbital fluctuations play an important role, and they 
also couple to spin fluctuations. Therefore, this behavior 
in the phase diagram reflects the stabilizing role of the 
joint spin-orbital fluctuations in the singlet phase, S — 0. 
When spin value increases, such fluctuations are partly 
damped and therefore intermediate spin values are here 
not realized. 

An interesting result was obtained for 116 cluster in the 
superexchange regime (at a ~ 0). Here the transition 
from the singlet to the 5 = 1 state occurs at 77 = 0.186. 
It is followed by the transition to the 5 = 2 state at 
77 = 0.231, and the final transition to the spin-polarized 
5 = 3 state takes place only at 77 = 0.242. Here the 
phases with intermediate spin values arise as a conse- 
quence of spin fluctuations that stabilize them in between 
the low-spin and high-spin states. A similar situation 
was found close to but not at the direct exchange point 
a — 1. In fact, spin fluctuations in the direct exchange 
limit concern only pairs of spins in the singlet 5 = 
phase, so a singlet-triplet transition for a single bond in- 
duces here the global transition to the 5 = 3 phase. Alto- 
gether, the transition to the high-spin phase occurs here 
at 77c — 0.188, in agreement with the classical expecta- 
tion, see also Eq. (|5.10|) in Sec. |Vl and this critical value 
is enhanced in the superexchange dominated regime due 
to spin-orbital fluctuations which are here stronger in the 
singlet phase. 

The phase diagrams obtained for three triangular clus- 
ters, N3, N6, and NIO, are shown in Fig. [T^ As for 
the hexagonal H6 cluster, the transition to the high-spin 
phase occurs for 77 ~ 0.18, but the critical value of 77 
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FIG. 12: (Color online) Phase diagram in the (q, rj) plane as 
obtained for triangular clusters with OBC: N3 (squares);^ N6 
(diamonds) and NIO (circles). The phase diagrams of N6 and 
NIO clusters contain also intermediate spin phases and the 
corresponding phase boundaries are shown as dashed lines 
(for the N6 cluster only near a = 1). 

is larger for two bigger clusters (N6 and NIO) than for 
the smallest N3 triangular cluster. One finds here once 
again a signature of the stabilizing role played by or- 
bital fluctuations in the low-spin phase. Particularly in 
the regime dominated by the superexchange, the onset 
of the high-spin phase occurs at values of rj which are 
higher than at a ~ 0.5 and increase from N3 {i] — 0.158) 
to larger clusters (r/ = 0.211 and i] = 0.198 for N6 and 
NIO clusters). In all three cases the transition occurs to 
the phases with maximal spin and no intermediate phases 
in between. 

Phases with intermediate spin values occur for two 
larger triangular clusters in the intermediate and direct 
exchange regime. We suggest that they are stabilized by 
orbital fluctuations which can couple to only partly po- 
larized spin subsystem and provide certain energy gain. 
While this feature is general, the actual range of stability 
of the phases with intermediate spin values depends on 
the cluster size. At a = 1 one finds for N6 a transition to 
S = 3 phase at a rather large value of 77 = 0.238, which 
demonstrates here particular stability of states with lower 
spin values, where more energy can be gained due to 
quantum fluctuations for certain orbital arrangements. 
However, in a larger NIO cluster this transition occurs at 
a lower value of ry = 0.202. 

When the phase diagrams obtained for the triangu- 
lar clusters (Fig. [T^ are compared with those for the 
clusters N7 and N9 with PBC (Fig. [T31), one finds that 
the transition to the high-spin states occurs in general for 
somewhat lower values of rj in the latter case. An extreme 
case here is N4 cluster with PBC, where a rather small 
value of a ~ 0.02 suffices to destabilize the singlet phase 
at a = 0. This peculiar result follows from the small size 
of this cluster which allows one to accommodate only two 
spin singlets when the orbital state is constrained to the 
pairs of orbitals supporting the superexchange processes. 



FIG. 13: (Color online) Phase diagram in the (q, 77) plane 
as obtained for clusters with PBC: N4 (squares);^ N7 (dia- 
monds) and N9 (circles). Ranges of stability of intermediate 
spin phases for N7 and N9 clusters are shown by thin lines 
and arrows. 



while in the high-spin S — 2 state these constraints are 
released and the orbital fiuctuations of different kind sta- 
bilize it. In contrast, a similar critical value in the range 
of 0.15 < 77 < 0.175 is found for the low-to-high spin 
transition in aU the clusters N4, N7, and N9 with PBC 
at a = 1. This common feature which is almost indepen- 
dent of the cluster size suggests that static dimer config- 
urations dominate in this case not only for the low-spin 
phase, ^'^ but also for the high-spin phase, where pairs of 
different orbitals are stable on individual bonds, see Fig. 

The proximity of critical values of 77 found for N7 and 
N9 clusters is very encouraging and suggests that one 
may expect the FM phase for rj > 0.16 in the thermody- 
namic limit, independently of the ratio of superexchange 
to direct exchange (i.e., on the actual value of a). In both 
clusters we have also found a range of stability for the 
phases with intermediate value of total spin which sug- 
gests that this transition is likely to be continuous, via 
weakly polarized FM states, also in the thermodynamic 
limit. It is remarkable, however, that intermediate spin 
states do not occur for the direct exchange interactions 
at a = 1 which demonstrates once again that quantum 
fluctuations do not play an important role in this case, 
and simple conflgurational averaging over available or- 
bital dimer configurations on the lattice suffices to un- 
derstand the magnetic transition described here. 



V. SPIN-ORBITAL ENTANGLEMENT 

A. Disentangled spin-orbital interactions 

When analyzing the phase diagrams for the clusters 
with PBC, we have emphasized the role played by or- 
bital and spin-orbital fiuctuations, as well as spin-orbital 
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entanglement in the GS. Now we shall present additional 
data to support this claim. In order to address this ques- 
tion we introduce an approximate treatment of the spin- 
orbital Hamiltonian which uses MF decoupling of spin 
and orbital variables on the bonds. "^^ Focusing on the 
magnetic interactions which concern here 5 = 1/2 quan- 
tum spins coupled by the SU(2) symmetric interaction, 
we rewrite the spin-orbital model Eq. (j2.3p in a gen- 
eral form<2^ 



(5.1) 



where the operators J^^^ and contain orbital pseu- 
dospin operators for a bond (ij) along the direction 7. 
This form is helpful as one can deduce the values of 
spin exchange constants directly from it when the or- 
bital operators are replaced by their averages in a par- 
ticular (ordered or disordered) state. In particular, it 
helped to understand the origin of magnetic interactions 
in LaMnOs,^^ where such a decoupling scheme may be 
well justified and gives predictions for the optical spectral 
weights^^ that agree with the experimental datai^ 

Here we introduce the MF procedure to simplify Eq. 
(15.11) as follows. 



(5.2) 



The operators for the bonds along two other lattice direc- 
tions can be obtained by permutations of {a, b, c} orbital 
indices. However, as all the bonds are equivalent, it suf- 
fices to consider the above operator J^j^ for a represen- 
tative bond to derive the exchange constant by averaging 
the orbital operators over the GS wave function |$o), 



Jmf = iWl^'^l'^o)- 



(5.4) 



Here the orbital fluctuation operators in the term oc 
a(l — a) contribute and couple different components 

of the wave function l^&o)- 

We also consider a simplified classical quantity. 



^MF = E 



;n|4^V)l("l<i>o)r 



(5.5) 



where the summation includes basis states {\n)} with all 
possible orbital configurations in the considered cluster. 
As the basis states are used for calculating the average 
in Eq. (|5.5p which is the same along all the bonds. This 
result may be derived from Eq. (|5.3p by neglecting the 
orbital dynamics, inter alia the terms cx a{l — a), and 
keeping only the diagonal orbital terms. One finds then 
the approximate form of the spin interaction. 



J, 



J{l-a) 



3ri 



»'2 



2^3 ( ^ 



ri - r2 



J a { : [uic (1 - rijc) + (1 - ^^ic) "jc] 



4 

2^2 + ra 



(5.6) 



The first term in Eq. (|5.2p is the spin model as introduced 
in Ref. [H, the second one is a purely orbital model, 
while the last one is a double counting correction term 
for the spin-orbital part of the Hamiltonian Eq. (|5.1I) . As 



' (c) 

an example we consider the orbital operator J7^|^- which 
stands as a coefficient of the Heisenberg spin interaction 
for a bond {ij) along the c axis. 



J, 



J(l - a) 



n - r2 



1 , 

- (riic + n^c) - 1 



n+r2 rj. + rj.+ 



Ja{ : [riic (1 - rijc) + (1 - "ic) «jc 



4 

2r2 + rs 



(5.3) 



Further simplification follows from an observation that 
for uniform electron distribution in the spin-orbital liquid 
state one finds the following averages for the relevant 
density and pseudospin operators: 



1 



{T^^T^^)^Q. (5.7) 



Using these expectation values in Eq. (|5.5p it follows, 

3ri +r2 + 2r3 



^Ja{r2-ri 



9 

2r2 + ra 



- ri + r2 



and one arrives at an analytic expression, 

-3ri -I- 5r2 -I- r^ 



27 



(5.8) 



(5.9) 



The classical exchange constant J^p Eq. (|5.9p is AF for 
small values of 77, in agreement with the results presented 
in Sees. IIII Bl and IllICi and changes sign at the critical 
value of Hund's exchange. 



rye = 0.188, 



(5.10) 
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FIG. 14: (Color online) Phase diagrams obtained for dis- 
entangled spin-orbital interactions following Eq. 1)5. 2|) ob- 
tained for N4, N7 and N9 clusters with PBC. No intermediate 
phases were found between low-spin (5 = 5min) and high-spin 
[S = <Smin) phases. 



where the present classical evaluation of the exchange 
constant predicts a transition to the FM phase. The 
present classical treatment suggests that this transition 
would occur simultaneously for all the bonds directly 
from the low-spin to high-spin state, with the maximal 
value of total spin S = N/2 for the cluster of N sites. 

As usually, the MF Hamiltonian Eq. (|5.2p implies a 
self-consistent solution of spin and orbital correlations. 
For instance, by solving a similar ID problem within 
the MF approach self-consistently, one finds dimerization 
in FM spin-orbital chains at finite temperature,'^^ Here 
we have applied the following iterative procedure for a 
system with spin and orbital interactions assumed to be 
isotropic on the triangular lattice: (i) for an initial value 

of spin scalar product I^Si-Sj^ we solved the orbital 

Hamiltonian, next (ii) the effective exchange constants 
given by Eq. (j5.4p were obtained, and (iii) they were 
used to determine the spin scalar product. This cycle 
was repeated until a self-consistent solution was found. 
We also used certain damping along the iteration process 
and enforced the symmetry of the considered clusters to 
accelerate the convergence as all the bonds are equivalent 
when PBC are used. 

The phase diagrams obtained following the above MF 
procedure for the clusters N4, N7 and N9 with PBC are 
presented in Fig. 1141 The applied procedure docs not 
use the total spin symmetry, so intermediate (partial) 
spin polarization of the cluster could not be resolved. 
Since we solve here the effective spin Hamiltonian inde- 
pendently of the orbital problem, the total spin state is 
determined entirely by the sign of the exchange constant 
Jmf- The spin correlations were found to be either nega- 
tive when Jmf > 0, indicating local AF (singlet-like) cor- 
relations, or were classical and FM, i.e., (^Si ■ Sj'^ = +j 
when Jmf < 0. One finds that the low-spin states are 



stable in a narrower range of the (a, 77) phase diagram 
for all three considered clusters, see Fig. [TH than when 
the exact diagonalization of the full Hamiltonian (|2.3p 
is performed (shown in Fig. I13p . In general, the phase 
boundary between the low-spin (5 — iSmin) and high-spin 
(S — iSmax) phase was found at a lower value of i] in each 
considered cluster than for the data extracted from exact 
diagonalization. 

Moreover, the superexchange and direct exchange 
cases are rather special and FM states appear in these 
limits already for infinitesimal values of 77. While the 
low-spin and high-spin state are degenerate in the direct 
exchange limit (at a = 1) for all three clusters shown 
in Fig. [131 in the superexchange case (at a = 1) such 
a degeneracy was found only for the N4 cluster. Both 
larger clusters N7 and N9 are FM already at r/ = in a 
range of small values of a: for a < 0.27 in case of N7 
and for a < 0.05 in case of N9 cluster. This behavior is 
surprising and suggests that the present MF procedure 
is unable to describe the present spin-orbital problem in 
a realistic way. We address this question in more detail 
in the next Sec. IV Bl 

B. Effective spin exchange constants 

In a spin system, as the one obtained from the spin- 
orbital model, intersite spin correlations follow the sign of 
the exchange constant, i.e., when the exchange constant 
changes sign and becomes negative, the spins align in 
the FM phase. The spin model extracted in the MF 
approximation from the spin-orbital model Eq. (j2.3p is 
given by 

= Jmf^5,;-^,, (5.11) 

where the exchange constant is given by Eq. (j5.4l) . 

Let us consider now once again the N7 cluster as a 
representative case to contrast the results obtained from 
the exact diagonalization and the present MF approach. 
The low-spin phase has S — 1/2, and the transition takes 
place to the S = 7/2 phase. Knowing that all the bonds 
are equivalent when PBC are used, and that each site has 
six neighbors, one finds that the spin correlation function 

for the low-spin S — 1/2 phase is (^Si ■ Sj^ = —3/28, as 

deduced from Eq. (jS.ip . The phase diagram shown in 
Fig. [13] may be understood as following from the change 
of sign of the MF exchange constant Eq. (j5.4p . Indeed, 
when the effective exchange constant is obtained for the 
entire (a, rj) plane, the onset of FM phase corresponds to 
the line Jmf = 0, see Fig. [rSTa). In general, the value 
of Jmf decreases with increasing 77 for any value of a, 
but positive values of the effective exchange constant are 
found only in a range of 77 values, if 0.27 < a < 1. This 
situation is unusual as the effective superexchange ob- 
tained in the MF approach favors FM spin order, even in 
the absence of Hund's exchange. It could be understood 
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FIG. 15: (Color online) Contour plots of the effective ex- 
change constant Jmf as obtained for N7 cluster with PBC 
from Eq. (|5.4|) : (a) within the MF calculation which includes 
orbital fluctuations, and (b) using exact GS found in exact di- 
agonalization. In case (a) the transition from low-spin to high- 
spin phase occurs when the exchange constant Jmf changes 
sign and becomes negative. Thick lines in (b) indicate the 
phase boundaries obtained between phases with increasing 
total spin value <S = 1/2, 3/2, 5/2 and 7/2 for increasing r\. 



as a consequence of strong orbital fluctuations in this 
regime which provide another mechanism of FM interac- 
tions playing an important role in the magnetic proper- 
ties of the RVO3 perovskites (i?=La,Y, etcetera) 

We remark that the phase diagram of N7 cluster (see 
Fig. I13p is quite different from the transition line shown 
in Fig. [Tsy a). Therefore, we conclude that the MF decou- 
pling scheme does not capture the essential features of the 
joint spin-orbital dynamics which stabilizes the low-spin 
phase in a broad regime of parameters. Furthermore, the 
results shown in Fig. llSf b) prove that spin transition in 
the spin-orbital model is not related to the apparent sign 
change in the exchange constant Jmf Eq. (j5.4p when it 
is calculated within the exact diagonalization approach. 
This also demonstrates that the frequently used MF pro- 
cedure to extract the spin exchange constants^^ might 
lead to uncontrolled results, particularly for frustrated 
systems with disorder in a form of spin-orbital liquid. 

Comparing the values of Jmf obtained using the MF 
procedure and the exact diagonalization [Figs. I15t a) and 
[T5f b). respectively], a qualitative change is found for the 



direct exchange case (a = 1), where positive values of 
Jmf extend now up to the transition point 77 ~ 0.156. 
Also for lower values of a the range of Jmf > is 
extended, particularly close to a = 0. However, the 
value of 77 corresponding to Jmf — systematically de- 
creases with decreasing a and is as low as i^c — 0.01 at 
a = 0. This behavior is in drastic contrast with the 
obtained transition from the low-spin to high-spin state, 
which occurs within the exact diagonalization method at 
a much higher value of ~ 0.16 in the entire range of 
a. Therefore, we conclude that the MF decoupling pro- 
cedure given in Eq. (|5.2p cannot be used in the present 
situation, similar as for the entangled states in the ID 
spin-orbital chains with active t2g orbitalsi^ 



VI. SUMMARY AND CONCLUSIONS 

The present study unravels dimer correlations present 
in the spin-orbital model on the triangular lattice. By 
considering finite clusters with open boundary conditions 
we have shown that such dimer correlations do exist when 
the interactions are either of the superexchange or of the 
direct exchange type. When the symmetry of the lattice 
is broken by the boundary in triangular clusters, spin 
singlets are favored on bonds which originate from cor- 
ner sites, and the orbital flavors adjust to the stronger 
channel, either to superexchange or to direct exchange. 
In fact, as different orbital states support these two types 
of magnetic interactions, the orbital distribution over the 
lattice is partly frustrated in the entire parameter regime, 
and this frustration contributes to the orbital disorder 
in the crossover regime, typically around a = 0.6. The 
most striking result here is the collapse of valence-bond 
states in the intermediate regime and the onset of spin- 
orbital liquid. Such a disordered state realized already 
in finite clusters with open boundary conditions in the 
regime of competing interactions suggests that it could 
extend over a broader regime of parameters when the 
geometry would not favor particular way of symmetry 
breaking in the spin-orbital space. 

Our study of the clusters with periodic boundary con- 
ditions provides indeed evidence that a quantum spin- 
orbital liquid phase is realized in the present spin- 
orbital model designed for t2g electrons on the geomet- 
rically frustrated triangular lattice. We would like to 
emphasize here that the frustrated lattice is necessary 
to remove tendency towards certain type of orbital order 
which could break the lattice symmetry in other cases, 
such as in the titanium or vanadium perovskites, and 
would support phases with long-range orbital accompa- 
nied by spin order of certain kind (usually following the 
Goodenough-Kanamori rules"^^). Thus, the spins behave 
here differently than in the spin model with Heisenberg 
interactions on the triangular lattice, and no spin order 
emerges when the spins couple to orbitals and both de- 
grees of freedom undergo joint quantum fluctuations. 

Although a mathematical proof is not possible, we pro- 
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vided, as we think, a rather complete and convincing ev- 
idence that the present spin-orbital model realizes a 
paradigm of spin- orbital liquid phase, and the order-out- 
of-disorder mechanism does not apply when the Hilbert 
space contains coupled spin and orbital sectors. Previous 
search for this quantum state of matter in other systems, 
particularly in LiNi02 where Cg orbitals are active on the 
triangular latticcj^S were unsuccessful^"''*^ After consid- 
ering the present model in more detail we suggest that 
the triple degeneracy of orbitals plays a crucial role in 
the onset of spin-orbital liquid, as the number of orbital 
flavors fits to the geometry of the triangular lattice. How- 
ever, one might expect that instead a three-sublattice or- 
dered state could arise, similar to the one known for spin 
system.™ We argue that the coupling to the spins plays 
here a very important role and spin-orbital entanglement 
is a characteristic feature of the disordered state found 
in the absence of Hund's exchange (at Jh = 0). 

We have shown than both geometry and spin-orbital 
interactions are the origin of frustration in the model 
under consideration. One may wonder in this context 
whether geometrical frustration on the triangular lattice 
enhances interaction frustration for spin-orbital models. 
Quite generally, spin-orbital models contain in principle 
more channels which can be used for relieving enhanced 
frustration so one might still expect that some kind of 
ordered states would emerge. We argue that this prob- 
lem is more subtle and its essence lies in the nature of 
spin-orbital entangled states. We have demonstrated by 
evaluating spin-orbital correlations that spin and orbital 
operators are entangled on the bonds and cannot be fac- 
torized. Under these circumstances important contribu- 
tions to the ground state energy arise from joint spin- 
orbital fluctuations. 

We have also found that Goodenough-Kanamori 
rules^ are not obeyed by spin and orbital bond corre- 
lations in some cases. This concerns in particular the su- 
perexchange regime where the low-spin phase is stabilized 
by them. Such entangled states play an important role 
in the vanadium perovskites at finite temperature,— 
are lead to topological constraints on the hole motion 
which couples simulataneosly to spin and optical exci- 
atation when states with entangled spin-orbital order are 
doped^S. We emphasize that the spin-orbital entangle- 
ment occurs here on the bonds, and should not be con- 
fused with spin-orbital singlets arising from strong on-site 
spin-orbit coupling which leads to spin-orbital liquid with 
local singlets,— and might also generate exotic phases, as 
shown recently in the case of spin 5 = 1/2 and a higher 
orbital quantum number (pseudospin) i = 1.^ 

By considering the magnetic transition to the ferro- 
magnetic phase, we have shown that a likely scenario for 



this transition is a crossover via the intermediate spin 
states before fully polarized ferromagnetic state sets in. 
This suggests that spin-orbital entangled states also play 
an important role in phases with partial spin polariza- 
tion, stabilizing them in the regime of transition towards 
the fully polarized ferromagnetic phase. Moreover, we de- 
tected a general principle concerning the applicability of 
effective spin models derived from spin-orbital Hamilto- 
nians. While this is a common practice nowadays which 
helps to understand and interpret the experimental data 
in systems with active orbital degrees of freedom,^ we 
presented evidence that even in case when magnetic ex- 
change constants can be accurately evaluated using the 
relevant orbital correlations, they might be inadequate 
to describe the magnetic ground state and excitations in 
such a system. This qualitative limitation could play a 
role particularly in disordered systems, where the physi- 
cal consequences of entangled spin-orbital states are more 
severe. 

In summary, the present spin-orbital model on the tri- 
angular lattice provides a beautiful example of highly 
frustrated interactions with the ground state dominated 
by: (i) quantum fluctuations, and (ii) spin-orbital entan- 
glement. It is for this reason that several naive expecta- 
tions which have their roots in classical expectations for 
complex spin systems do not apply, and it is even impos- 
sible to describe correctly the interactions for the mag- 
netic degrees of freedom by decoupling them from the 
orbital ones. Although lattice distortions and coupling 
between the planes of a triangular lattice might desta- 
bilize the spin-orbital liquid found here, we hope that 
its experimental example could be established by future 
experimental studies. The spin-orbital disordered state 
provides a challenge both for the theory and for the ex- 
periment to find a way of describing magnetic excitations 
arising in a spin-orbital liquid phase. 
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